{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "62549efe",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Overview\n",
    "\n",
    "Here, we'll see how to define a `Kork` CodeChain that can invoke external python functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "79fd8e41",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    },
    "nbsphinx": "hidden",
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": [
     "remove_cell"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "import sys\n",
    "\n",
    "sys.path.insert(0, \"../\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ff38ea21-b562-4c13-be0f-c394843bfe39",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import langchain\n",
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.llms import OpenAI\n",
    "\n",
    "from kork import CodeChain"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e0b5e44-217a-4e03-ac8b-c94ac632c98a",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "tags": []
   },
   "source": [
    "## Foreign Functions\n",
    "\n",
    "Let's create a list of python functions that we want the chain to be able to invoke.\n",
    "\n",
    "Information about these functions will be added into the prompt to let the LLM know it can use them.\n",
    "\n",
    "The functions will also be added as foregin functions into the kork interpreter environment, so the interpreter can actually use them!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "0ba46c52-662e-4b91-9494-de07150dd5cd",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def output_with_matplotlib(output: str) -> None:\n",
    "    \"\"\"Function that will output a plot using matplotlib.\"\"\"\n",
    "    if not isinstance(output, str):\n",
    "        raise ValueError(f\"LLM failed to produce a string!\")\n",
    "\n",
    "    import matplotlib.pyplot as plt\n",
    "\n",
    "    fig, ax = plt.subplots()\n",
    "    ax.axis(\"off\")\n",
    "    ax.text(0.5, 0.5, output, ha=\"center\", va=\"center\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "329ec257-2ad3-4eb3-a734-5e144510f38f",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def foo(what_to_print: str) -> str:\n",
    "    \"\"\"A function that something to the screen\"\"\"\n",
    "    print(\"Invoking a custom user defined function\")\n",
    "    try:\n",
    "        import matplotlib\n",
    "\n",
    "        output_with_matplotlib(what_to_print)\n",
    "    except ImportError:\n",
    "        print(\"Matplotlib missing so using print\")\n",
    "        print(what_to_print)\n",
    "    print(\"Finished invoking a custom user defined function\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "870d37c3-7659-4de3-9590-f6851c48ece9",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import math"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "0c9a7c56-dce9-4dda-86c5-2fd2cd92984c",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "foreign_functions = [foo, math.pow, math.log2, math.log10, math.sqrt]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8eced1e7-4ea4-4454-bb5d-8c65eac6ff33",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Examples\n",
    "\n",
    "Use examples to teach the llm how to work in the target problem domain. Later in the docs, we'll see how to customize the entire prompt.\n",
    "\n",
    "If the programs invoke functinos, make sure that the function names match exactly the names of the foreign functions provided (no validation here currently).\n",
    "\n",
    "We can specify the programs in text format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "cc620e8c-c235-4ee4-b264-4611c1b8eebb",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "examples = [\n",
    "    (\"calculate the sqrt of 2\", \"let result = pow(2, 0.5)\"),\n",
    "    (\"2*5 + 1\", \"let result = 2 * 5 + 1\"),\n",
    "    (\"1.3e-3\", \"let result = 1.3 * pow(10, -3)\"),\n",
    "    (\"2**5\", \"let result = pow(2, 5)\"),\n",
    "    (\"calculate log of 2\", \"let result = log2(2)\"),\n",
    "    (\"set the value of result to the string hello\", 'let result = \"hello\";'),\n",
    "    (\n",
    "        \"every day i eat 3 donuts. how many donuts do i eat during the week\",\n",
    "        \"let days_in_week = 7; let result = days_in_week * 3;\",\n",
    "    ),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78784491-8945-466c-8670-96d01682ed45",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "But before we can feed them into the chain, we need to convert them into an abstract syntax tree (ast):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "4524bfea-8b3d-4722-b908-fe2a0e0f8d66",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from kork.parser import parse\n",
    "\n",
    "examples_in_ast = [(query, parse(code)) for query, code in examples]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17b7b765-f80e-4b54-87ad-fbf144ae2223",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now the first example is composed of a 2-tuple of query and a corresponding program expressed in `Kork` AST."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "7b841410-e2ed-4747-b76e-d8957b6ee35c",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('calculate the sqrt of 2',\n",
       " Program(stmts=(VarDecl(name='result', value=FunctionCall(name='pow', args=(Literal(value=2), Literal(value=0.5)))),)))"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "examples_in_ast[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "473a66b8-5939-4aa0-a954-04a7fc337506",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## CodeChain\n",
    "\n",
    "The inputs into the chains will be explained in detail later in the documentation.\n",
    "\n",
    "For now, let's run the code chain and look at the generated prompt and the output from the chain!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "db474c2a-eb99-4260-9bd8-da8b6eec5564",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "llm = OpenAI(\n",
    "    model_name=\"text-davinci-003\",\n",
    "    temperature=0,\n",
    "    max_tokens=2000,\n",
    "    frequency_penalty=0,\n",
    "    presence_penalty=0,\n",
    "    top_p=1.0,\n",
    "    verbose=True,\n",
    ")\n",
    "\n",
    "chain = CodeChain.from_defaults(\n",
    "    llm=llm,\n",
    "    examples=examples_in_ast,\n",
    "    context=foreign_functions,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07698ce6-47d1-4757-a486-9e9af4cd8d28",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Let's see if we can the LLM to invoke our print function?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "d97cc291-14be-40b2-944d-e267238f1086",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Invoking a custom user defined function\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAGFCAYAAABg2vAPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAYcklEQVR4nO3ceZRWdf3A8c84pMimCAhoOqTIEiGipWD9AgkFM3LBJU4uIXhAyKLQc9RM8VBikIp0cMPA3FMhKzM1QSrJZBQXVHAUBTdcIjcQQYbv748OcxxZhF92+Nnn9Tpn/rjPvfd7v/cyzPOe57nPVJRSSgAAaW2ztScAAGxdYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBiAf9Ptt98e7du3j8rKyhg1atTWns4n4jvf+U4cccQRW3san5j/tvOBT5oYSOzVV1+N0047LfbYY4/YbrvtYrfddosBAwbEzJkzN3uMa665Jnbcccf/3CQ/BYYNGxZHH310vPjiizF27NitPZ0tsnjx4qioqIhHH310a0+Fjfi4kFm5cmWcd9550aFDh9huu+2iZcuWccwxx8STTz5Zb7sxY8bEPvvss8ljtWvXLiZOnPjvT5pPHTGQ1OLFi2O//faLWbNmxYQJE2L+/Plx1113xUEHHRQjR47c2tP7WKtXr97aU4iIiOXLl8frr78e/fr1i1122SWaNm26tadEIqtWrYq+ffvG1KlT4yc/+UnU1NTEnXfeGWvWrIkDDjgg/v73v2/tKfIpIQaSGjFiRFRUVMTcuXNj4MCB0aFDh+jSpUv88Ic/rPcD5OKLL46uXbtG48aNY7fddosRI0bE8uXLIyJi9uzZMXjw4Hj77bejoqIiKioqYsyYMRERcdlll8Vee+0VDRs2jNatW8fRRx+90bksW7YsBg0aFLvuums0atQounbtGjfddFO9bXr37h3f/e53Y9SoUdGyZcvo169fREQ88cQTceihh0aTJk2idevWccIJJ8Q//vGPjR5ryZIlMWDAgGjevHk0btw4unTpEnfeeWfd+j//+c+x//77x3bbbRdt27aNM888M9asWbPBsWbPnl335N+nT5+oqKiI2bNnb/A3sIkTJ0a7du3qltf9tvfzn/882rZtGy1atIiRI0fGBx98ULdNu3bt4oILLoiTTz45mjZtGrvvvntcddVVdev79OkT3/3ud+sd54033ohtt912s1/d+dznPhcREd27d4+Kioro3bt3vfWbmt+qVavi9NNPj1133TUaN24cBxxwQMyePTsiIlasWBHNmjWL2267rd54t99+ezRu3DjefffdDc6nd+/ecdppp8WoUaOiefPm0bp165gyZUqsWLEiBg8eHE2bNo327dvHH//4x7p9amtrY8iQIfG5z30utt9+++jYsWNceumlmzzv6urqaNWqVfzsZz+LiIi33norhg4dGq1atYpmzZpFnz594rHHHtvo/uteUbnlllvif/7nf2L77bePL33pS1FTUxPV1dXxxS9+MZo0aRKHHnpovPHGG/X2vfrqq6Nz587RsGHD6NSpU1x22WWbnOumTJw4MR544IG444474thjj42qqqrYf//9Y/r06dG5c+cYMmRIlFL+z+OTSCGdZcuWlYqKinLBBRd87LaXXHJJmTVrVnn++efLzJkzS8eOHcupp55aSill1apVZeLEiaVZs2Zl6dKlZenSpeXdd98t1dXVpbKystx4441l8eLFZd68eeXSSy/d6DFeeumlMmHChPLII4+URYsWlUmTJpXKysry4IMP1m3Tq1ev0qRJk3LGGWeUhQsXloULF5Y333yztGrVqpx11lllwYIFZd68eeXggw8uBx100EaPddhhh5WDDz64PP7442XRokXl97//ffnzn/9cN49GjRqVESNGlAULFpTf/OY3pWXLluW8887b4FirVq0qTz/9dImIMn369LJ06dKyatWqct5555Vu3bqtdx2rqqrqlk866aTSrFmzMnz48LJgwYLy+9//vjRq1KhcddVVddtUVVWVnXbaqUyePLk888wzZdy4cWWbbbYpCxcuLKWUcsMNN5TmzZuX999/v26fiy++uLRr166sXbt2o9fgw+bOnVsiotx7771l6dKlZdmyZZs9v6FDh5YDDzyw/OUvfynPPvtsmTBhQtluu+1KTU1NKaWUU045pXz961+vd7xvfvOb5cQTT9zofHr16lWaNm1axo4dW2pqasrYsWNLZWVlOfTQQ8tVV11VampqyqmnnlpatGhRVqxYUUopZfXq1eXcc88t1dXV5bnnnivXX399adSoUfn1r39d73offvjhpZRSZs6cWXbYYYdy5ZVX1q3v27dvGTBgQKmuri41NTVl9OjRpUWLFnXX46Oef/75EhGlU6dO5a677ipPPfVU6dGjR9lvv/1K7969y/3331/mzZtX2rdvX4YPH1633/XXX1/atm1bpk+fXp577rkyffr0stNOO5Vrrrlmo9fkw3P/qL333rsccsghG1x3ww03lIgojzzySCmlbPD78qOqqqrKJZdcsslt+O8kBhJ68MEHS0SUGTNmbPG+t956a2nRokXd8rRp08oOO+xQb5vp06eXZs2alXfeeef/PMfDDjusjB49um65V69epXv37vW2GTt27Ho/CF988cUSEeXpp5/e4Lhdu3YtY8aM2eC6s88+u3Ts2LHeE+nkyZNLkyZNSm1t7Qb3efPNN0tElPvuu6/usc2NgaqqqrJmzZq6x4455phy3HHH1S1XVVWV448/vm557dq1Zeeddy6XX355KaWUlStXlubNm9d70tt77703en4bsu5Jbd0TxubOb8mSJaWysrK8/PLL9fb72te+Vs4666xSyr++zyorK8srr7xSSinltddeKw0aNCizZ8/e6Hx69epVvvKVr9Qtr1mzpjRu3LiccMIJdY8tXbq0RER54IEHNjrOyJEjy8CBA+udz+GHH15mzJhRmjRpUm6++ea6dX/9619Ls2bN6kVVKaXsueee9YLhw9Zdt6uvvrrusZtuuqlERJk5c2bdY+PGjSsdO3asN+aNN95Yb6yxY8eWnj17bvRcNhUDDRs2LN///vc3uG7evHklIuq+P8QAm9Jg67wewdZUtuBlw3vvvTfGjRsXCxcujHfeeSfWrFkT77//frz33nvRqFGjDe5z8MEHR1VVVeyxxx7Rv3//6N+/fxx55JEb3b62tjYuuOCCuOWWW+Lll1+O1atXx6pVq9bbfr/99qu3/Nhjj8V9990XTZo0WW/MRYsWRYcOHdZ7/Hvf+16ceuqpcc8990Tfvn1j4MCBsffee0dExIIFC6Jnz55RUVFRt/2Xv/zlWL58ebz00kux++67b/pibaEuXbpEZWVl3XLbtm1j/vz59bZZN7eIiIqKimjTpk28/vrrERHRsGHDOOGEE2Lq1Klx7LHHxrx58+KJJ56I3/3ud//x+c2fPz9qa2vXu8arVq2KFi1aRETE/vvvH126dIlf/epXceaZZ8b1118fVVVV8dWvfnWTx/3wOVdWVkaLFi2ia9eudY+1bt06IqLuOkRETJ48OaZOnRovvPBCrFy5MlavXr3eWzUPPvhg3HHHHXHbbbfVuyHvsccei+XLl9fNe52VK1fGokWLNnuu6+b10bmum+eKFSti0aJFMWTIkDjllFPqtlmzZk3ssMMOmzzOpmzJ/2fYGDGQ0F577RUVFRWxcOHCTW63ePHi+MY3vhGnnnpq/PSnP42ddtop7r///hgyZEisXr16o0/uTZs2jXnz5sXs2bPjnnvuiXPPPTfGjBkT1dXVG/zkwYQJE+LSSy+NiRMn1t2fMGrUqPVuEmzcuHG95eXLl8eAAQPq3vf9sLZt225wbkOHDo1+/frFH/7wh7jnnnti3LhxcdFFF8Vpp522yWuxJbbZZpv1fkB/+L32dT7zmc/UW66oqIi1a9du0TZDhw6NffbZJ1566aWYNm1a9OnTJ6qqqv7dU/jYYy9fvjwqKyvj4YcfrhcMEVEvzoYOHRqTJ0+OM888M6ZNmxaDBw+uF1ube9wPP7Zu/3Vzufnmm+P000+Piy66KHr27BlNmzaNCRMmxIMPPlhvnD333DNatGgRU6dOjcMOO6xuzOXLl0fbtm3r7nf4sI/7pMyG5vXRxz58zSIipkyZEgcccEC9cT56DTdXhw4dYsGCBRtct+7xDUUxfJQbCBPaaaedol+/fjF58uRYsWLFeuvfeuutiIh4+OGHY+3atXHRRRdFjx49okOHDvHKK6/U23bbbbeN2tra9cZo0KBB9O3bN8aPHx+PP/54LF68OGbNmrXB+cyZMycOP/zwOP7446Nbt26xxx57RE1Nzceex7777htPPvlktGvXLtq3b1/v66Ph8GG77bZbDB8+PGbMmBGjR4+OKVOmRERE586d44EHHqj3RD5nzpxo2rRpfPazn/3Y+azTqlWrePXVV+uN85/66F7Xrl3ji1/8YkyZMiVuvPHGOPnkk7do/2233TYiYoP/hpvSvXv3qK2tjddff329a9+mTZu67Y4//vhYsmRJTJo0KZ566qk46aSTtug4m2POnDlx4IEHxogRI6J79+7Rvn37Df5G37Jly5g1a1Y8++yzceyxx9YF2r777huvvvpqNGjQYL1zadmy5Sc2z9atW8cuu+wSzz333HrHWXcj55b61re+Fffee+96NzuuXbs2Lrnkkvj85z8f3bp12+zxFi9e/F/ztzLYMmIgqcmTJ0dtbW3dncfPPPNMLFiwICZNmhQ9e/aMiIj27dvHBx98EL/4xS/iueeei+uuuy6uuOKKeuO0a9culi9fHjNnzox//OMf8d5778Udd9wRkyZNikcffTSWLFkS1157baxduzY6duy4wbnstdde8ac//Sn+9re/xYIFC2LYsGHx2muvfew5jBw5Mv75z3/GoEGDorq6OhYtWhR33313DB48eKNPbqNGjYq77747nn/++Zg3b17cd9990blz54j41ycsXnzxxTjttNNi4cKF8dvf/jbOO++8+OEPfxjbbLP5/1V69+4db7zxRowfPz4WLVoUkydPrnf3+ydt6NChceGFF0YpJY488sgt2nfnnXeO7bffPu6666547bXX4u23396s/Tp06BDf/va348QTT4wZM2bE888/H3Pnzo1x48bFH/7wh7rtmjdvHkcddVScccYZccghh2xRVG2uvfbaKx566KG4++67o6amJn784x9HdXX1BrfdeeedY9asWbFw4cIYNGhQrFmzJvr27Rs9e/aMI444Iu65555YvHhx/O1vf4sf/ehH8dBDD32icz3//PNj3LhxMWnSpKipqYn58+fHtGnT4uKLL97kfm+//XY8+uij9b5efPHF+MEPfhD7779/DBgwIG699dZ44YUXorq6OgYOHBgLFiyIX/7yl/VeiVm5cuV643w4nDp16hTXXnvtJ3rOfDqIgaT22GOPmDdvXhx00EExevTo+MIXvhAHH3xwzJw5My6//PKIiOjWrVtcfPHF8bOf/Sy+8IUvxA033BDjxo2rN86BBx4Yw4cPj+OOOy5atWoV48ePjx133DFmzJgRffr0ic6dO8cVV1wRN910U3Tp0mWDcznnnHNi3333jX79+kXv3r2jTZs2m/XX4nbZZZeYM2dO1NbWxiGHHBJdu3aNUaNGxY477rjRJ+/a2toYOXJkdO7cOfr37x8dOnSo+2jXrrvuGnfeeWfMnTs3unXrFsOHD48hQ4bEOeecswVX9l+vMFx22WUxefLk6NatW8ydOzdOP/30LRpjSwwaNCgaNGgQgwYNioYNG9ZbN2bMmHofafyoBg0axKRJk+LKK6+MXXbZJQ4//PDNPu60adPixBNPjNGjR0fHjh3jiCOOiOrq6vXurVj3ttKWvmqxuYYNGxZHHXVUHHfccXHAAQfEsmXLYsSIERvdvk2bNjFr1qyYP39+fPvb3461a9fGnXfeGV/96ldj8ODB0aFDh/jWt74VS5YsqbsP4JMydOjQuPrqq2PatGnRtWvX6NWrV1xzzTUf+8rA7Nmzo3v37vW+zj///GjYsGHMmjUrTjzxxDj77LOjffv20b9//6isrIy///3v0aNHj3rj1NTUrDfOsGHD6tY//fTT8c9//vMTPWc+HSqKu0/gU23x4sWx5557RnV1dey777711p100klRUVER11xzzdaZXERcd9118YMf/CBeeeWVurclgP9f3EAIn1IffPBBLFu2LM4555zo0aPHeiFQSonZs2fH/fffv1Xm995778XSpUvjwgsvjGHDhgkB+H/M2wTwKTVnzpxo27ZtVFdXr3cvR8S/7mRfsmRJ7LbbblthdhHjx4+PTp06RZs2beKss87aKnMANo+3CQAgOa8MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJITAwCQnBgAgOTEAAAkJwYAIDkxAADJiQEASE4MAEByYgAAkhMDAJCcGACA5MQAACQnBgAgOTEAAMmJAQBITgwAQHJiAACSEwMAkJwYAIDkxAAAJCcGACA5MQAAyYkBAEhODABAcmIAAJL7Xx7q3rv0IW/eAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Finished invoking a custom user defined function\n"
     ]
    }
   ],
   "source": [
    "code_result = chain(\n",
    "    inputs={\n",
    "        \"query\": \"write a program to output a funny sentence about a cat to the screen\"\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ae795e5-52d5-44e7-bd16-f14c5f62e2b2",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "---------\n",
    "\n",
    "Here's the code that was written by the LLM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "a3dc26b1-5814-4099-af07-5e88e41e25ff",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from kork.display import display_code_result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "5579c744-760b-4146-8976-7f1892f6c893",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style type=\"text/css\">\n",
       "#T_65019_row0_col0, #T_65019_row0_col1 {\n",
       "  text-align: left;\n",
       "}\n",
       "</style>\n",
       "<table id=\"T_65019\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_65019_level0_col0\" class=\"col_heading level0 col0\" >query</th>\n",
       "      <th id=\"T_65019_level0_col1\" class=\"col_heading level0 col1\" >code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_65019_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "      <td id=\"T_65019_row0_col0\" class=\"data row0 col0\" >write a program to output a<br/>funny sentence about a cat to<br/>the screen</td>\n",
       "      <td id=\"T_65019_row0_col1\" class=\"data row0 col1\" ><style>pre { line-height: 125%; }<br/>td.linenos .normal { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }<br/>span.linenos { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }<br/>td.linenos .special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }<br/>span.linenos.special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }<br/>.output_html .hll { background-color: #ffffcc }<br/>.output_html { background: #f8f8f8; }<br/>.output_html .c { color: #3D7B7B; font-style: italic } /* Comment */<br/>.output_html .err { border: 1px solid #FF0000 } /* Error */<br/>.output_html .k { color: #008000; font-weight: bold } /* Keyword */<br/>.output_html .o { color: #666666 } /* Operator */<br/>.output_html .ch { color: #3D7B7B; font-style: italic } /* Comment.Hashbang */<br/>.output_html .cm { color: #3D7B7B; font-style: italic } /* Comment.Multiline */<br/>.output_html .cp { color: #9C6500 } /* Comment.Preproc */<br/>.output_html .cpf { color: #3D7B7B; font-style: italic } /* Comment.PreprocFile */<br/>.output_html .c1 { color: #3D7B7B; font-style: italic } /* Comment.Single */<br/>.output_html .cs { color: #3D7B7B; font-style: italic } /* Comment.Special */<br/>.output_html .gd { color: #A00000 } /* Generic.Deleted */<br/>.output_html .ge { font-style: italic } /* Generic.Emph */<br/>.output_html .gr { color: #E40000 } /* Generic.Error */<br/>.output_html .gh { color: #000080; font-weight: bold } /* Generic.Heading */<br/>.output_html .gi { color: #008400 } /* Generic.Inserted */<br/>.output_html .go { color: #717171 } /* Generic.Output */<br/>.output_html .gp { color: #000080; font-weight: bold } /* Generic.Prompt */<br/>.output_html .gs { font-weight: bold } /* Generic.Strong */<br/>.output_html .gu { color: #800080; font-weight: bold } /* Generic.Subheading */<br/>.output_html .gt { color: #0044DD } /* Generic.Traceback */<br/>.output_html .kc { color: #008000; font-weight: bold } /* Keyword.Constant */<br/>.output_html .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */<br/>.output_html .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */<br/>.output_html .kp { color: #008000 } /* Keyword.Pseudo */<br/>.output_html .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */<br/>.output_html .kt { color: #B00040 } /* Keyword.Type */<br/>.output_html .m { color: #666666 } /* Literal.Number */<br/>.output_html .s { color: #BA2121 } /* Literal.String */<br/>.output_html .na { color: #687822 } /* Name.Attribute */<br/>.output_html .nb { color: #008000 } /* Name.Builtin */<br/>.output_html .nc { color: #0000FF; font-weight: bold } /* Name.Class */<br/>.output_html .no { color: #880000 } /* Name.Constant */<br/>.output_html .nd { color: #AA22FF } /* Name.Decorator */<br/>.output_html .ni { color: #717171; font-weight: bold } /* Name.Entity */<br/>.output_html .ne { color: #CB3F38; font-weight: bold } /* Name.Exception */<br/>.output_html .nf { color: #0000FF } /* Name.Function */<br/>.output_html .nl { color: #767600 } /* Name.Label */<br/>.output_html .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */<br/>.output_html .nt { color: #008000; font-weight: bold } /* Name.Tag */<br/>.output_html .nv { color: #19177C } /* Name.Variable */<br/>.output_html .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */<br/>.output_html .w { color: #bbbbbb } /* Text.Whitespace */<br/>.output_html .mb { color: #666666 } /* Literal.Number.Bin */<br/>.output_html .mf { color: #666666 } /* Literal.Number.Float */<br/>.output_html .mh { color: #666666 } /* Literal.Number.Hex */<br/>.output_html .mi { color: #666666 } /* Literal.Number.Integer */<br/>.output_html .mo { color: #666666 } /* Literal.Number.Oct */<br/>.output_html .sa { color: #BA2121 } /* Literal.String.Affix */<br/>.output_html .sb { color: #BA2121 } /* Literal.String.Backtick */<br/>.output_html .sc { color: #BA2121 } /* Literal.String.Char */<br/>.output_html .dl { color: #BA2121 } /* Literal.String.Delimiter */<br/>.output_html .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */<br/>.output_html .s2 { color: #BA2121 } /* Literal.String.Double */<br/>.output_html .se { color: #AA5D1F; font-weight: bold } /* Literal.String.Escape */<br/>.output_html .sh { color: #BA2121 } /* Literal.String.Heredoc */<br/>.output_html .si { color: #A45A77; font-weight: bold } /* Literal.String.Interpol */<br/>.output_html .sx { color: #008000 } /* Literal.String.Other */<br/>.output_html .sr { color: #A45A77 } /* Literal.String.Regex */<br/>.output_html .s1 { color: #BA2121 } /* Literal.String.Single */<br/>.output_html .ss { color: #19177C } /* Literal.String.Symbol */<br/>.output_html .bp { color: #008000 } /* Name.Builtin.Pseudo */<br/>.output_html .fm { color: #0000FF } /* Name.Function.Magic */<br/>.output_html .vc { color: #19177C } /* Name.Variable.Class */<br/>.output_html .vg { color: #19177C } /* Name.Variable.Global */<br/>.output_html .vi { color: #19177C } /* Name.Variable.Instance */<br/>.output_html .vm { color: #19177C } /* Name.Variable.Magic */<br/>.output_html .il { color: #666666 } /* Literal.Number.Integer.Long */</style><div class=\"highlight\"><pre><span></span><span class=\"k\">var</span><span class=\"w\"> </span><span class=\"n\">result</span><span class=\"w\"> </span><span class=\"o\">=</span><span class=\"w\"> </span><span class=\"n\">foo</span><span class=\"p\">(</span><span class=\"s2\">&quot;Cats are so funny, they make me LOL!&quot;</span><span class=\"p\">)</span><br/></pre></div></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display_code_result(code_result, columns=[\"query\", \"code\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b6706b6-339d-4e13-89fc-94a05a8bd977",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "```{hint}\n",
    "The LLM successfully invoked a custom user function called `foo` which printed to the screen. 🤔\n",
    "\n",
    "Feel free to experiment and create functions that create images, play sounds etc. \n",
    "\n",
    "The chain will be able to invoke any functions that you provide it.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c9c4132-aa6e-4b3c-8df5-254dc8974576",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "```{note}\n",
    "\n",
    "The LLM may return `var result = 0` at the end of the program.\n",
    "\n",
    "The examples and prompt bias it to end in a variable declaration statement.\n",
    "\n",
    "The prompt and examples can be customized to change this behavior.\n",
    "```\n",
    "\n",
    "```{caution}\n",
    "\n",
    "You may see failures even in simple tasks like this one. See the guidelines section.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97a49b50-d402-4d4d-bdb2-9b806d43cc5c",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## The prompt\n",
    "\n",
    "Let's turn on verbose mode and take a look at the prompt that gets generated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "8064e12e-0f4b-49ed-829c-1874177743de",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "langchain.verbose = True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8e86e02-8a65-4d4e-9f0c-5569af9ec160",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "And let's try a more complex sentence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "dd104f07-5d40-4985-a313-2782fa826bc7",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3mYou are programming in a language called \"😼\".\n",
      "\n",
      "You are an expert programmer and must follow the instructions below exactly.\n",
      "\n",
      "Your goal is to translate a user query into a corresponding and valid 😼\n",
      "program.\n",
      "\n",
      "You have access to the following external functions:\n",
      "\n",
      "```😼\n",
      "extern fn foo(what_to_print: str) -> str // A function that something to the screen\n",
      "extern fn pow(x: Any, y: Any) -> Any // Return x**y (x to the power of y).\n",
      "extern fn log2(x: Any) -> Any // Return the base 2 logarithm of x.\n",
      "extern fn log10(x: Any) -> Any // Return the base 10 logarithm of x.\n",
      "extern fn sqrt(x: Any) -> Any // Return the square root of x.\n",
      "```\n",
      "\n",
      "\n",
      "Do not assume that any other functions except for the ones listed above exist.\n",
      "\n",
      "Wrap the program in <code> and </code> tags.\n",
      "\n",
      "Store the solution to the query in a variable called \"result\".\n",
      "\n",
      "Here is a sample valid program:\n",
      "\n",
      "<code>\n",
      "var x = 1 # Assign 1 to the variable x\n",
      "var result = 1 + 2 # Calculate the sum of 1 + 2 and assign to result\n",
      "var result = x # Assign the value of x to result\n",
      "</code>\n",
      "\n",
      "Guidelines:\n",
      "- Do not use operators, instead invoke appropriate external functions.\n",
      "- Do not declare functions, do not use loops, do not use conditionals.\n",
      "- Solve the problem only using variable declarations and function invocations.\n",
      "\n",
      "Begin!\n",
      "Input: ```text\n",
      "calculate the sqrt of 2\n",
      "```\n",
      "\n",
      "Output: <code>var result = pow(2, 0.5)</code>\n",
      "\n",
      "Input: ```text\n",
      "2*5 + 1\n",
      "```\n",
      "\n",
      "Output: <code>var result = 2 * 5 + 1</code>\n",
      "\n",
      "Input: ```text\n",
      "1.3e-3\n",
      "```\n",
      "\n",
      "Output: <code>var result = 1.3 * pow(10, -3)</code>\n",
      "\n",
      "Input: ```text\n",
      "2**5\n",
      "```\n",
      "\n",
      "Output: <code>var result = pow(2, 5)</code>\n",
      "\n",
      "Input: ```text\n",
      "calculate log of 2\n",
      "```\n",
      "\n",
      "Output: <code>var result = log2(2)</code>\n",
      "\n",
      "Input: ```text\n",
      "set the value of result to the string hello\n",
      "```\n",
      "\n",
      "Output: <code>var result = \"hello\"</code>\n",
      "\n",
      "Input: ```text\n",
      "every day i eat 3 donuts. how many donuts do i eat during the week\n",
      "```\n",
      "\n",
      "Output: <code>var days_in_week = 7\n",
      "var result = days_in_week * 3</code>\n",
      "\n",
      "Input: ```text\n",
      "An ecoli cell doubles every 30 mins in a test tube. How long will it take 1 cell to become 16 cells? Use variable declarations and function invocations.\n",
      "```\n",
      "\n",
      "Output:\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "query = \"An ecoli cell doubles every 30 mins in a test tube. How long will it take 1 cell to become 16 cells? Use variable declarations and function invocations.\"\n",
    "\n",
    "code_result = chain(inputs={\"query\": query})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e140cbe-aa0d-4f25-9411-be76d9341d7c",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "The llm output was:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "4ab38d38-d272-4664-ac03-aefdebb22d7d",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style type=\"text/css\">\n",
       "#T_03237_row0_col0, #T_03237_row0_col1, #T_03237_row0_col2, #T_03237_row0_col3, #T_03237_row0_col4, #T_03237_row0_col5, #T_03237_row0_col6 {\n",
       "  text-align: left;\n",
       "}\n",
       "</style>\n",
       "<table id=\"T_03237\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_03237_level0_col0\" class=\"col_heading level0 col0\" >query</th>\n",
       "      <th id=\"T_03237_level0_col1\" class=\"col_heading level0 col1\" >code</th>\n",
       "      <th id=\"T_03237_level0_col2\" class=\"col_heading level0 col2\" >result</th>\n",
       "      <th id=\"T_03237_level0_col3\" class=\"col_heading level0 col3\" >expected</th>\n",
       "      <th id=\"T_03237_level0_col4\" class=\"col_heading level0 col4\" >correct</th>\n",
       "      <th id=\"T_03237_level0_col5\" class=\"col_heading level0 col5\" >errors</th>\n",
       "      <th id=\"T_03237_level0_col6\" class=\"col_heading level0 col6\" >raw</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_03237_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "      <td id=\"T_03237_row0_col0\" class=\"data row0 col0\" >An ecoli cell doubles every 30<br/>mins in a test tube. How long<br/>will it take 1 cell to become<br/>16 cells? Use variable<br/>declarations and function<br/>invocations.</td>\n",
       "      <td id=\"T_03237_row0_col1\" class=\"data row0 col1\" ><style>pre { line-height: 125%; }<br/>td.linenos .normal { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }<br/>span.linenos { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }<br/>td.linenos .special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }<br/>span.linenos.special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }<br/>.output_html .hll { background-color: #ffffcc }<br/>.output_html { background: #f8f8f8; }<br/>.output_html .c { color: #3D7B7B; font-style: italic } /* Comment */<br/>.output_html .err { border: 1px solid #FF0000 } /* Error */<br/>.output_html .k { color: #008000; font-weight: bold } /* Keyword */<br/>.output_html .o { color: #666666 } /* Operator */<br/>.output_html .ch { color: #3D7B7B; font-style: italic } /* Comment.Hashbang */<br/>.output_html .cm { color: #3D7B7B; font-style: italic } /* Comment.Multiline */<br/>.output_html .cp { color: #9C6500 } /* Comment.Preproc */<br/>.output_html .cpf { color: #3D7B7B; font-style: italic } /* Comment.PreprocFile */<br/>.output_html .c1 { color: #3D7B7B; font-style: italic } /* Comment.Single */<br/>.output_html .cs { color: #3D7B7B; font-style: italic } /* Comment.Special */<br/>.output_html .gd { color: #A00000 } /* Generic.Deleted */<br/>.output_html .ge { font-style: italic } /* Generic.Emph */<br/>.output_html .gr { color: #E40000 } /* Generic.Error */<br/>.output_html .gh { color: #000080; font-weight: bold } /* Generic.Heading */<br/>.output_html .gi { color: #008400 } /* Generic.Inserted */<br/>.output_html .go { color: #717171 } /* Generic.Output */<br/>.output_html .gp { color: #000080; font-weight: bold } /* Generic.Prompt */<br/>.output_html .gs { font-weight: bold } /* Generic.Strong */<br/>.output_html .gu { color: #800080; font-weight: bold } /* Generic.Subheading */<br/>.output_html .gt { color: #0044DD } /* Generic.Traceback */<br/>.output_html .kc { color: #008000; font-weight: bold } /* Keyword.Constant */<br/>.output_html .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */<br/>.output_html .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */<br/>.output_html .kp { color: #008000 } /* Keyword.Pseudo */<br/>.output_html .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */<br/>.output_html .kt { color: #B00040 } /* Keyword.Type */<br/>.output_html .m { color: #666666 } /* Literal.Number */<br/>.output_html .s { color: #BA2121 } /* Literal.String */<br/>.output_html .na { color: #687822 } /* Name.Attribute */<br/>.output_html .nb { color: #008000 } /* Name.Builtin */<br/>.output_html .nc { color: #0000FF; font-weight: bold } /* Name.Class */<br/>.output_html .no { color: #880000 } /* Name.Constant */<br/>.output_html .nd { color: #AA22FF } /* Name.Decorator */<br/>.output_html .ni { color: #717171; font-weight: bold } /* Name.Entity */<br/>.output_html .ne { color: #CB3F38; font-weight: bold } /* Name.Exception */<br/>.output_html .nf { color: #0000FF } /* Name.Function */<br/>.output_html .nl { color: #767600 } /* Name.Label */<br/>.output_html .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */<br/>.output_html .nt { color: #008000; font-weight: bold } /* Name.Tag */<br/>.output_html .nv { color: #19177C } /* Name.Variable */<br/>.output_html .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */<br/>.output_html .w { color: #bbbbbb } /* Text.Whitespace */<br/>.output_html .mb { color: #666666 } /* Literal.Number.Bin */<br/>.output_html .mf { color: #666666 } /* Literal.Number.Float */<br/>.output_html .mh { color: #666666 } /* Literal.Number.Hex */<br/>.output_html .mi { color: #666666 } /* Literal.Number.Integer */<br/>.output_html .mo { color: #666666 } /* Literal.Number.Oct */<br/>.output_html .sa { color: #BA2121 } /* Literal.String.Affix */<br/>.output_html .sb { color: #BA2121 } /* Literal.String.Backtick */<br/>.output_html .sc { color: #BA2121 } /* Literal.String.Char */<br/>.output_html .dl { color: #BA2121 } /* Literal.String.Delimiter */<br/>.output_html .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */<br/>.output_html .s2 { color: #BA2121 } /* Literal.String.Double */<br/>.output_html .se { color: #AA5D1F; font-weight: bold } /* Literal.String.Escape */<br/>.output_html .sh { color: #BA2121 } /* Literal.String.Heredoc */<br/>.output_html .si { color: #A45A77; font-weight: bold } /* Literal.String.Interpol */<br/>.output_html .sx { color: #008000 } /* Literal.String.Other */<br/>.output_html .sr { color: #A45A77 } /* Literal.String.Regex */<br/>.output_html .s1 { color: #BA2121 } /* Literal.String.Single */<br/>.output_html .ss { color: #19177C } /* Literal.String.Symbol */<br/>.output_html .bp { color: #008000 } /* Name.Builtin.Pseudo */<br/>.output_html .fm { color: #0000FF } /* Name.Function.Magic */<br/>.output_html .vc { color: #19177C } /* Name.Variable.Class */<br/>.output_html .vg { color: #19177C } /* Name.Variable.Global */<br/>.output_html .vi { color: #19177C } /* Name.Variable.Instance */<br/>.output_html .vm { color: #19177C } /* Name.Variable.Magic */<br/>.output_html .il { color: #666666 } /* Literal.Number.Integer.Long */</style><div class=\"highlight\"><pre><span></span><span class=\"k\">var</span><span class=\"w\"> </span><span class=\"n\">cell_count</span><span class=\"w\"> </span><span class=\"o\">=</span><span class=\"w\"> </span><span class=\"mi\">1</span><br/><span class=\"k\">var</span><span class=\"w\"> </span><span class=\"n\">target_cell_count</span><span class=\"w\"> </span><span class=\"o\">=</span><span class=\"w\"> </span><span class=\"mi\">16</span><br/><span class=\"k\">var</span><span class=\"w\"> </span><span class=\"n\">doubling_time</span><span class=\"w\"> </span><span class=\"o\">=</span><span class=\"w\"> </span><span class=\"mi\">30</span><br/><span class=\"k\">var</span><span class=\"w\"> </span><span class=\"n\">result</span><span class=\"w\"> </span><span class=\"o\">=</span><span class=\"w\"> </span><span class=\"n\">log2</span><span class=\"p\">(</span><span class=\"n\">target_cell_count</span><span class=\"w\"> </span><span class=\"o\">/</span><span class=\"w\"> </span><span class=\"n\">cell_count</span><span class=\"p\">)</span><span class=\"w\"> </span><span class=\"o\">*</span><span class=\"w\"> </span><span class=\"n\">doubling_time</span><br/></pre></div></td>\n",
       "      <td id=\"T_03237_row0_col2\" class=\"data row0 col2\" >120.0</td>\n",
       "      <td id=\"T_03237_row0_col3\" class=\"data row0 col3\" >120.0</td>\n",
       "      <td id=\"T_03237_row0_col4\" class=\"data row0 col4\" >✅</td>\n",
       "      <td id=\"T_03237_row0_col5\" class=\"data row0 col5\" >[]</td>\n",
       "      <td id=\"T_03237_row0_col6\" class=\"data row0 col6\" > &lt;code&gt;var cell_count = 1<br/>var target_cell_count = 16<br/>var doubling_time = 30<br/>var result = log2(target_cell_count / cell_count) * doubling_time&lt;/code&gt;</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display_code_result(code_result, expected_answer=120.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcc33990-9d2f-4492-834b-c5a886884e77",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## The Environment\n",
    "\n",
    "After running the chain, we have access to state of the environment after the interpreter finished running the code.\n",
    "\n",
    "We can fetch the `result` variable from the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "a9bf5af7-9796-411f-b593-7bad1caa908f",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "kork.environment.Environment"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(code_result[\"environment\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b579a41-056e-444b-a5ad-e80f9256d5b6",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "The result of the calculation for the e-coli cell is stored in the `result` symbol"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "6ccb5677-403f-433a-9a5b-58016d330704",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "120.0\n"
     ]
    }
   ],
   "source": [
    "print(code_result[\"environment\"].get_symbol(\"result\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a4da309-c7eb-44c5-a3bb-f7962396caf0",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Relativistic Kinetic Energy\n",
    "\n",
    "We can try to carry out a more complex physics calculation.\n",
    "\n",
    "```{important}\n",
    "\n",
    "LLMs can struggle with longer and more complex tasks.\n",
    "\n",
    "Guidelines:\n",
    "\n",
    "1. Use a better LLM\n",
    "2. Customize the prompt for your task\n",
    "3. Provide examples\n",
    "4. Provide foreign functions that are simple and include type annotations and short doc-strings.\n",
    "```\n",
    "\n",
    "```{attention}\n",
    "\n",
    "`Kork` is very much a work in progress. \n",
    "\n",
    "It's not yet doing a 2nd pass with an LLM to help correct bugs in the generated program, which\n",
    "will likley help improve results.\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "0aff6e68-04f5-4086-9e55-7d851f006a14",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3mYou are programming in a language called \"😼\".\n",
      "\n",
      "You are an expert programmer and must follow the instructions below exactly.\n",
      "\n",
      "Your goal is to translate a user query into a corresponding and valid 😼\n",
      "program.\n",
      "\n",
      "You have access to the following external functions:\n",
      "\n",
      "```😼\n",
      "extern fn foo(what_to_print: str) -> str // A function that something to the screen\n",
      "extern fn pow(x: Any, y: Any) -> Any // Return x**y (x to the power of y).\n",
      "extern fn log2(x: Any) -> Any // Return the base 2 logarithm of x.\n",
      "extern fn log10(x: Any) -> Any // Return the base 10 logarithm of x.\n",
      "extern fn sqrt(x: Any) -> Any // Return the square root of x.\n",
      "```\n",
      "\n",
      "\n",
      "Do not assume that any other functions except for the ones listed above exist.\n",
      "\n",
      "Wrap the program in <code> and </code> tags.\n",
      "\n",
      "Store the solution to the query in a variable called \"result\".\n",
      "\n",
      "Here is a sample valid program:\n",
      "\n",
      "<code>\n",
      "var x = 1 # Assign 1 to the variable x\n",
      "var result = 1 + 2 # Calculate the sum of 1 + 2 and assign to result\n",
      "var result = x # Assign the value of x to result\n",
      "</code>\n",
      "\n",
      "Guidelines:\n",
      "- Do not use operators, instead invoke appropriate external functions.\n",
      "- Do not declare functions, do not use loops, do not use conditionals.\n",
      "- Solve the problem only using variable declarations and function invocations.\n",
      "\n",
      "Begin!\n",
      "Input: ```text\n",
      "calculate the sqrt of 2\n",
      "```\n",
      "\n",
      "Output: <code>var result = pow(2, 0.5)</code>\n",
      "\n",
      "Input: ```text\n",
      "2*5 + 1\n",
      "```\n",
      "\n",
      "Output: <code>var result = 2 * 5 + 1</code>\n",
      "\n",
      "Input: ```text\n",
      "1.3e-3\n",
      "```\n",
      "\n",
      "Output: <code>var result = 1.3 * pow(10, -3)</code>\n",
      "\n",
      "Input: ```text\n",
      "2**5\n",
      "```\n",
      "\n",
      "Output: <code>var result = pow(2, 5)</code>\n",
      "\n",
      "Input: ```text\n",
      "calculate log of 2\n",
      "```\n",
      "\n",
      "Output: <code>var result = log2(2)</code>\n",
      "\n",
      "Input: ```text\n",
      "set the value of result to the string hello\n",
      "```\n",
      "\n",
      "Output: <code>var result = \"hello\"</code>\n",
      "\n",
      "Input: ```text\n",
      "every day i eat 3 donuts. how many donuts do i eat during the week\n",
      "```\n",
      "\n",
      "Output: <code>var days_in_week = 7\n",
      "var result = days_in_week * 3</code>\n",
      "\n",
      "Input: ```text\n",
      "Let's calculate the kinetic energy of a relativistic electron moving at 0.9999c in MeV. \n",
      "First list all relevant physical constants and conver them into an appropriate units.\n",
      "Now, calculate the lorentz factor.\n",
      "Then subtract one from it, and multiply the result by the rest mass of an electron in MeV.\n",
      "Make sure to use the listed foreign functions and no binary arithemtic operators.\n",
      "```\n",
      "\n",
      "Output:\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "query = \"\"\"\\\n",
    "Let's calculate the kinetic energy of a relativistic electron moving at 0.9999c in MeV. \n",
    "First list all relevant physical constants and conver them into an appropriate units.\n",
    "Now, calculate the lorentz factor.\n",
    "Then subtract one from it, and multiply the result by the rest mass of an electron in MeV.\n",
    "Make sure to use the listed foreign functions and no binary arithemtic operators.\"\"\"\n",
    "\n",
    "code_result = chain(inputs={\"query\": query})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "e8ca83e2-2e38-4a27-bd3f-54bbecb77383",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "expected_answer = 0.511 * ((1 - 0.9999**2) ** (-0.5) - 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d580e46d-4a68-48e3-9bf6-5890b94676b3",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "```{caution}\n",
    "\n",
    "Success rate of this calculation is higher than 5%, but definitely lower than 80%. Huge variation due to underlying LLM!\n",
    "\n",
    "This is a cherry picked run when things work as expected. A 2nd pass with an LLM may be needed to get more robust.\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "91ab0dd8-beee-4e2d-aba8-7d0c1591cd05",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style type=\"text/css\">\n",
       "#T_6d48c_row0_col0, #T_6d48c_row0_col1, #T_6d48c_row0_col2, #T_6d48c_row0_col3, #T_6d48c_row0_col4, #T_6d48c_row0_col5, #T_6d48c_row0_col6 {\n",
       "  text-align: left;\n",
       "}\n",
       "</style>\n",
       "<table id=\"T_6d48c\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_6d48c_level0_col0\" class=\"col_heading level0 col0\" >query</th>\n",
       "      <th id=\"T_6d48c_level0_col1\" class=\"col_heading level0 col1\" >code</th>\n",
       "      <th id=\"T_6d48c_level0_col2\" class=\"col_heading level0 col2\" >result</th>\n",
       "      <th id=\"T_6d48c_level0_col3\" class=\"col_heading level0 col3\" >expected</th>\n",
       "      <th id=\"T_6d48c_level0_col4\" class=\"col_heading level0 col4\" >correct</th>\n",
       "      <th id=\"T_6d48c_level0_col5\" class=\"col_heading level0 col5\" >errors</th>\n",
       "      <th id=\"T_6d48c_level0_col6\" class=\"col_heading level0 col6\" >raw</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_6d48c_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "      <td id=\"T_6d48c_row0_col0\" class=\"data row0 col0\" >Let&#x27;s calculate the<br/>kinetic energy of a<br/>relativistic electron moving<br/>at 0.9999c in MeV.  First list<br/>all relevant physical<br/>constants and conver them into<br/>an appropriate units. Now,<br/>calculate the lorentz factor.<br/>Then subtract one from it, and<br/>multiply the result by the<br/>rest mass of an electron in<br/>MeV. Make sure to use the<br/>listed foreign functions and<br/>no binary arithemtic<br/>operators.</td>\n",
       "      <td id=\"T_6d48c_row0_col1\" class=\"data row0 col1\" ><style>pre { line-height: 125%; }<br/>td.linenos .normal { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }<br/>span.linenos { color: inherit; background-color: transparent; padding-left: 5px; padding-right: 5px; }<br/>td.linenos .special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }<br/>span.linenos.special { color: #000000; background-color: #ffffc0; padding-left: 5px; padding-right: 5px; }<br/>.output_html .hll { background-color: #ffffcc }<br/>.output_html { background: #f8f8f8; }<br/>.output_html .c { color: #3D7B7B; font-style: italic } /* Comment */<br/>.output_html .err { border: 1px solid #FF0000 } /* Error */<br/>.output_html .k { color: #008000; font-weight: bold } /* Keyword */<br/>.output_html .o { color: #666666 } /* Operator */<br/>.output_html .ch { color: #3D7B7B; font-style: italic } /* Comment.Hashbang */<br/>.output_html .cm { color: #3D7B7B; font-style: italic } /* Comment.Multiline */<br/>.output_html .cp { color: #9C6500 } /* Comment.Preproc */<br/>.output_html .cpf { color: #3D7B7B; font-style: italic } /* Comment.PreprocFile */<br/>.output_html .c1 { color: #3D7B7B; font-style: italic } /* Comment.Single */<br/>.output_html .cs { color: #3D7B7B; font-style: italic } /* Comment.Special */<br/>.output_html .gd { color: #A00000 } /* Generic.Deleted */<br/>.output_html .ge { font-style: italic } /* Generic.Emph */<br/>.output_html .gr { color: #E40000 } /* Generic.Error */<br/>.output_html .gh { color: #000080; font-weight: bold } /* Generic.Heading */<br/>.output_html .gi { color: #008400 } /* Generic.Inserted */<br/>.output_html .go { color: #717171 } /* Generic.Output */<br/>.output_html .gp { color: #000080; font-weight: bold } /* Generic.Prompt */<br/>.output_html .gs { font-weight: bold } /* Generic.Strong */<br/>.output_html .gu { color: #800080; font-weight: bold } /* Generic.Subheading */<br/>.output_html .gt { color: #0044DD } /* Generic.Traceback */<br/>.output_html .kc { color: #008000; font-weight: bold } /* Keyword.Constant */<br/>.output_html .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */<br/>.output_html .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */<br/>.output_html .kp { color: #008000 } /* Keyword.Pseudo */<br/>.output_html .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */<br/>.output_html .kt { color: #B00040 } /* Keyword.Type */<br/>.output_html .m { color: #666666 } /* Literal.Number */<br/>.output_html .s { color: #BA2121 } /* Literal.String */<br/>.output_html .na { color: #687822 } /* Name.Attribute */<br/>.output_html .nb { color: #008000 } /* Name.Builtin */<br/>.output_html .nc { color: #0000FF; font-weight: bold } /* Name.Class */<br/>.output_html .no { color: #880000 } /* Name.Constant */<br/>.output_html .nd { color: #AA22FF } /* Name.Decorator */<br/>.output_html .ni { color: #717171; font-weight: bold } /* Name.Entity */<br/>.output_html .ne { color: #CB3F38; font-weight: bold } /* Name.Exception */<br/>.output_html .nf { color: #0000FF } /* Name.Function */<br/>.output_html .nl { color: #767600 } /* Name.Label */<br/>.output_html .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */<br/>.output_html .nt { color: #008000; font-weight: bold } /* Name.Tag */<br/>.output_html .nv { color: #19177C } /* Name.Variable */<br/>.output_html .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */<br/>.output_html .w { color: #bbbbbb } /* Text.Whitespace */<br/>.output_html .mb { color: #666666 } /* Literal.Number.Bin */<br/>.output_html .mf { color: #666666 } /* Literal.Number.Float */<br/>.output_html .mh { color: #666666 } /* Literal.Number.Hex */<br/>.output_html .mi { color: #666666 } /* Literal.Number.Integer */<br/>.output_html .mo { color: #666666 } /* Literal.Number.Oct */<br/>.output_html .sa { color: #BA2121 } /* Literal.String.Affix */<br/>.output_html .sb { color: #BA2121 } /* Literal.String.Backtick */<br/>.output_html .sc { color: #BA2121 } /* Literal.String.Char */<br/>.output_html .dl { color: #BA2121 } /* Literal.String.Delimiter */<br/>.output_html .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */<br/>.output_html .s2 { color: #BA2121 } /* Literal.String.Double */<br/>.output_html .se { color: #AA5D1F; font-weight: bold } /* Literal.String.Escape */<br/>.output_html .sh { color: #BA2121 } /* Literal.String.Heredoc */<br/>.output_html .si { color: #A45A77; font-weight: bold } /* Literal.String.Interpol */<br/>.output_html .sx { color: #008000 } /* Literal.String.Other */<br/>.output_html .sr { color: #A45A77 } /* Literal.String.Regex */<br/>.output_html .s1 { color: #BA2121 } /* Literal.String.Single */<br/>.output_html .ss { color: #19177C } /* Literal.String.Symbol */<br/>.output_html .bp { color: #008000 } /* Name.Builtin.Pseudo */<br/>.output_html .fm { color: #0000FF } /* Name.Function.Magic */<br/>.output_html .vc { color: #19177C } /* Name.Variable.Class */<br/>.output_html .vg { color: #19177C } /* Name.Variable.Global */<br/>.output_html .vi { color: #19177C } /* Name.Variable.Instance */<br/>.output_html .vm { color: #19177C } /* Name.Variable.Magic */<br/>.output_html .il { color: #666666 } /* Literal.Number.Integer.Long */</style><div class=\"highlight\"><pre><span></span><span class=\"k\">var</span><span class=\"w\"> </span><span class=\"n\">c</span><span class=\"w\"> </span><span class=\"o\">=</span><span class=\"w\"> </span><span class=\"mi\">299792458</span><span class=\"w\"> </span><span class=\"o\">//</span><span class=\"w\"> </span><span class=\"n\">Speed</span><span class=\"w\"> </span><span class=\"n\">of</span><span class=\"w\"> </span><span class=\"n\">light</span><span class=\"w\"> </span><span class=\"ow\">in</span><span class=\"w\"> </span><span class=\"n\">m</span><span class=\"o\">/</span><span class=\"n\">s</span><br/><span class=\"k\">var</span><span class=\"w\"> </span><span class=\"n\">rest_mass_electron</span><span class=\"w\"> </span><span class=\"o\">=</span><span class=\"w\"> </span><span class=\"mf\">0.511</span><span class=\"w\"> </span><span class=\"o\">//</span><span class=\"w\"> </span><span class=\"n\">Rest</span><span class=\"w\"> </span><span class=\"n\">mass</span><span class=\"w\"> </span><span class=\"n\">of</span><span class=\"w\"> </span><span class=\"n\">an</span><span class=\"w\"> </span><span class=\"n\">electron</span><span class=\"w\"> </span><span class=\"ow\">in</span><span class=\"w\"> </span><span class=\"n\">MeV</span><br/><span class=\"k\">var</span><span class=\"w\"> </span><span class=\"n\">speed</span><span class=\"w\"> </span><span class=\"o\">=</span><span class=\"w\"> </span><span class=\"mf\">0.9999</span><span class=\"w\"> </span><span class=\"o\">*</span><span class=\"w\"> </span><span class=\"n\">c</span><span class=\"w\"> </span><span class=\"o\">//</span><span class=\"w\"> </span><span class=\"n\">Speed</span><span class=\"w\"> </span><span class=\"n\">of</span><span class=\"w\"> </span><span class=\"n\">the</span><span class=\"w\"> </span><span class=\"n\">electron</span><span class=\"w\"> </span><span class=\"ow\">in</span><span class=\"w\"> </span><span class=\"n\">m</span><span class=\"o\">/</span><span class=\"n\">s</span><br/><span class=\"k\">var</span><span class=\"w\"> </span><span class=\"n\">lorentz_factor</span><span class=\"w\"> </span><span class=\"o\">=</span><span class=\"w\"> </span><span class=\"mi\">1</span><span class=\"w\"> </span><span class=\"o\">/</span><span class=\"w\"> </span><span class=\"nb\">sqrt</span><span class=\"p\">(</span><span class=\"mi\">1</span><span class=\"w\"> </span><span class=\"o\">-</span><span class=\"w\"> </span><span class=\"nb\">pow</span><span class=\"p\">(</span><span class=\"n\">speed</span><span class=\"p\">,</span><span class=\"w\"> </span><span class=\"mi\">2</span><span class=\"p\">)</span><span class=\"w\"> </span><span class=\"o\">/</span><span class=\"w\"> </span><span class=\"nb\">pow</span><span class=\"p\">(</span><span class=\"n\">c</span><span class=\"p\">,</span><span class=\"w\"> </span><span class=\"mi\">2</span><span class=\"p\">))</span><span class=\"w\"> </span><span class=\"o\">//</span><span class=\"w\"> </span><span class=\"n\">Calculate</span><span class=\"w\"> </span><span class=\"n\">the</span><span class=\"w\"> </span><span class=\"n\">lorentz</span><span class=\"w\"> </span><span class=\"n\">factor</span><br/><span class=\"k\">var</span><span class=\"w\"> </span><span class=\"n\">result</span><span class=\"w\"> </span><span class=\"o\">=</span><span class=\"w\"> </span><span class=\"p\">(</span><span class=\"n\">lorentz_factor</span><span class=\"w\"> </span><span class=\"o\">-</span><span class=\"w\"> </span><span class=\"mi\">1</span><span class=\"p\">)</span><span class=\"w\"> </span><span class=\"o\">*</span><span class=\"w\"> </span><span class=\"n\">rest_mass_electron</span><span class=\"w\"> </span><span class=\"o\">//</span><span class=\"w\"> </span><span class=\"n\">Calculate</span><span class=\"w\"> </span><span class=\"n\">the</span><span class=\"w\"> </span><span class=\"n\">kinetic</span><span class=\"w\"> </span><span class=\"n\">energy</span><span class=\"w\"> </span><span class=\"ow\">in</span><span class=\"w\"> </span><span class=\"n\">MeV</span><br/></pre></div></td>\n",
       "      <td id=\"T_6d48c_row0_col2\" class=\"data row0 col2\" >35.62305988140826</td>\n",
       "      <td id=\"T_6d48c_row0_col3\" class=\"data row0 col3\" >35.62305988142832</td>\n",
       "      <td id=\"T_6d48c_row0_col4\" class=\"data row0 col4\" >✅</td>\n",
       "      <td id=\"T_6d48c_row0_col5\" class=\"data row0 col5\" >[]</td>\n",
       "      <td id=\"T_6d48c_row0_col6\" class=\"data row0 col6\" > &lt;code&gt;var c = 299792458 // Speed of light in m/s<br/>var rest_mass_electron = 0.511 // Rest mass of an electron in MeV<br/>var speed = 0.9999 * c // Speed of the electron in m/s<br/>var lorentz_factor = 1 / sqrt(1 - pow(speed, 2) / pow(c, 2)) // Calculate the lorentz factor<br/>var result = (lorentz_factor - 1) * rest_mass_electron // Calculate the kinetic energy in MeV&lt;/code&gt;</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "display_code_result(code_result, expected_answer=expected_answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c24b6c1-91cb-4f7a-b6f3-76a2d619e2bd",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Calculations with LLMs\n",
    "\n",
    "As LLMs get better and better they will better approximate the correct answer (without having to even specify an algorithm).\n",
    "\n",
    "You can try this exercise with GPT-4 or chat gpt. With an appropriate prompt and model, the results can look pretty reasonable!\n",
    "\n",
    "Alternatively, you can compare the results with langchain math chains!\n",
    "\n",
    "As an experiment, you could use an LLM to generate a calculation plan for a given scientific question and then use a code chain to translate the instructions into a program.\n",
    "\n",
    "\n",
    "```{caution}\n",
    "\n",
    "Similarily to humans it looks like the LLMs can struggle a bit with unit conversions. 😹\n",
    "\n",
    "But the knowledge is there, so maybe we just need to be clever enough!\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "7f9848fd-7134-443a-9084-55f624eb8785",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "The relativistic kinetic energy of an electron moving at 0.999999 c is 7.979 MeV.\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    llm.generate(\n",
    "        prompts=[\n",
    "            \"What is the relativistic kinetic energy of an electron moving at 0.999999 c expressed in MeV?\"\n",
    "        ]\n",
    "    )\n",
    "    .generations[0][0]\n",
    "    .text\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "7b408840-ab98-4adb-89a5-a56ea71dc7e9",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "The relativistic kinetic energy of an electron moving at 0.9999 c can be calculated using the equation:\n",
      "\n",
      "KE = (γ - 1)mc2\n",
      "\n",
      "Where γ is the Lorentz factor, m is the mass of the electron, and c is the speed of light.\n",
      "\n",
      "Step 1: Calculate the Lorentz factor\n",
      "\n",
      "γ = 1/√(1 - v2/c2)\n",
      "\n",
      "γ = 1/√(1 - (0.9999)2)\n",
      "\n",
      "γ = 1/√(1 - 0.99980001)\n",
      "\n",
      "γ = 1/√(0.00019999)\n",
      "\n",
      "γ = 1/0.0141\n",
      "\n",
      "γ = 70.7107\n",
      "\n",
      "Step 2: Calculate the relativistic kinetic energy\n",
      "\n",
      "KE = (γ - 1)mc2\n",
      "\n",
      "KE = (70.7107 - 1)mc2\n",
      "\n",
      "KE = 69.7107mc2\n",
      "\n",
      "KE = 69.7107 x (9.109 x 10-31 kg) x (3 x 108 m/s)2\n",
      "\n",
      "KE = 1.945 x 10-13 J\n",
      "\n",
      "Step 3: Convert Joules to MeV\n",
      "\n",
      "1 MeV = 1.602 x 10-13 J\n",
      "\n",
      "KE = 1.945 x 10-13 J / 1.602 x 10-13 J/MeV\n",
      "\n",
      "KE = 1.21 MeV\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    llm.generate(\n",
    "        prompts=[\n",
    "            \"What is the relativistic kinetic energy of an electron moving at 0.9999 c expressed in MeV explain step by step calculation?\"\n",
    "        ]\n",
    "    )\n",
    "    .generations[0][0]\n",
    "    .text\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "0396dd7e-16d5-4ef7-90b5-f1cef128298f",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "1. Determine the speed of the electron in terms of the speed of light (c). In this case, the electron is moving at 0.9999c.\n",
      "\n",
      "2. Calculate the relativistic factor (γ) using the equation γ = 1/√(1-v2/c2).\n",
      "\n",
      "3. Calculate the relativistic kinetic energy (Ek) using the equation Ek = γmc2 - mc2.\n",
      "\n",
      "4. Substitute the values for γ and c into the equation and solve for Ek.\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    llm.generate(\n",
    "        prompts=[\n",
    "            \"Generate step by step instructions to calculate the relativistic kinetic energy of an electron moving at 0.9999c?\"\n",
    "        ]\n",
    "    )\n",
    "    .generations[0][0]\n",
    "    .text\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
